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Abstract. Starting from a general operator representation in the time- frequency 
domain, this paper addresses the problem of approximating linear operators by 
operators that arc diagonal or band-diagonal with respect to Gabor frames. A 
characterization of operators that can be realized as Gabor multipliers is given 
and necessary conditions for the existence of (Hilbcrt-Schmidt) optimal Gabor 
multiplier approximations arc discussed and an cfhcicnt method for the calcula- 
tion of an operator's best approximation by a Gabor multiplier is derived. The 
spreading function of Gabor multipliers yields new error estimates for these ap- 
proximations. Generalizations (multiple Gabor multipliers) are introduced for 
better approximation of overspread operators. The Riesz property of the pro- 
jection operators involved in generalized Gabor multipliers is characterized, and 
a method for obtaining an operator's best approximation by a multiple Gabor 
multiplier is suggested. Finally, it is shown that in certain situations, general- 
ized Gabor multipliers reduce to a finite sum of regular Gabor multipliers with 
adapted windows. 



1. Introduction 

The goal of time-frequency analysis is to provide efficient representations for 
functions or distributions in terms of decompositions such as 

AeA 

Here, / is expanded as a weighted sum of atoms h\ well localized in both time 
and frequency domains. The time-frequency coefficients {f^gx) characterize the 
function under investigation, and a synthesis map usually allows the reconstruction 
of the original function /. 

Concrete applications can be found mostly in signal analysis and processing (see [H 
[9l [35] and references therein) , but recent works in different areas such as numerical 
analysis may also be mentioned (see for example [HI [19] and references therein). 

Time-frequency analysis of operators, originating in the work on communication 
channels of Bello [S], Kailath [21] and Zadeh [ID], has enjoyed increasing interest 
during the last few years, [23 [Ml [Ml [2]- Efficient time-frequency operator rep- 
resentation is a challenging task, and often the intuitively appealing approach of 
operator approximation by modification of the time-frequency coefficients before 
reconstruction is the method of choice. If the modification of the coefficients is 
confined to be multiplicative, this approach leads to the model of time-frequency 
multipliers, as discussed in Section 12. 4[ The class of operators that may be well 

Date: September 16, 2008. 

2000 Mathematics Subject Classification. 47B38,47G30,94A12,65F20. 

Key words and phrases. Operator approximation, generalized Gabor multipliers, spreading 
function, twisted convolution. 

The first author has been supported by the WWTF project MA07-G25. 



2 



MONIKA DORFLER AND BRUNO TORRESANI 



represented by time-frequency multipliers depends on the choice of the parame- 
ters involved and is restricted to operators performing only small time-shifts or 
modulations. 

The work in this paper is inspired by a general operator representation in the 
time-frequency domain via a twisted convolution. It turns out, that this representa- 
tion, respecting the underlying structure of the Heisenberg group, has an interesting 
connection to the so-called spreading function representation of operators. An op- 
erator's spreading function comprises the amount of time-shifts and modulations, 
i.e. of time-frequency-shifts, effected by the operator. Its investigation is hence 
decisive in the study of time-frequency multipliers and their generalizations. Al- 
though no direct discretization of the continuous representation by an operator's 
spreading function is possible, the twisted convolution turns out to play an impor- 
tant role in the generalizations of time-frequency multipliers. In the main section of 
this article, we introduce a general model for multiple Gabor multipliers (MGM), 
which uses several synthesis windows simultaneously. Thus, by jointly adapting the 
respective masks, more general operators may be well-represented than by regular 
Gabor multipliers. Specifying to a separable mask in the modification of time- 
frequency coefficients within MGM, as well as a specific sampling lattice for the 
synthesis windows, it turns out, that the MGM reduces to one or the sum of a 
finite number of regular Gabor multipliers with adapted synthesis windows. 
For the sake of generality, most statements are given in a Gelfand-triple, rather 
than a pure Hilbert space setting. This choice bears several advantages. First of 
all, many important operators and signals may not be described in a Hilbert-space 
setting, starting from simple operators as the identity. Furthermore, by using dis- 
tributions, continuous and discrete concepts may be considered together. Finally, 
the Gelfand-triple setting often allows for short-cut proofs of statements formulated 
in a general context. 

This paper is organized as follows. The next section gives a review of the time- 
frequency plane and the corresponding continuous and discrete transforms. We 
then introduce the concept of Gelfand triples, which will allow us to consider op- 
erators beyond the Hilbert-Schmidt framework. The section closes with the im- 
portant statement on operator-representation in the time-frequency domain via 
twisted convolution with an operator's spreading function. Section 12.41 introduces 
time-frequency multipliers and gives a criterion for their ability to approximate 
linear operators. A fast method for the calculation of an operator's best approx- 
imation by a Gabor multiplier in Hilbert-Schmidt sense is suggested. Section 3 
introduces generalizations of Gabor multipliers. The operators in the construction 
of MGM are investigated and a criterion for their Riesz basis property in the space 
of Hilbert-Schmidt operators is given. We mention some connections to classical 
Gabor frames. A numerical example concludes the discussion of general MGM. 
In the final section, TST (twisted spline type) spreading functions are introduced. 
It is shown, that under certain conditions, a MGM reduces to a regular Gabor 
multiplier with an adapted window or a finite sum of regular multipliers with the 
same mask and adapted windows. 

2. Operators from the Time-frequency point of view 

Whenever one is interested in time-localized frequency information in a signal or 
operator, one is naturally led to the notion of the time-frequeny plane, which, in 
turn, is closely related to the Weyl-Heisenberg group. 
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2.1. Preliminaries: the time-frequency plane. The starting point of our op- 
erator analysis is the so-called spreading function operator representation. This 
operator representation expresses linear operators as a sum (in a sense to be spec- 
ified below) of time-frequency shifts 7i{b,iy) = MyTb. Here, the translation and 
modulation operators are defined as 

T,f{t) = fit - b) , MJ{t) = e^'^^'fit) , / e L2(M) . 

These (unitary) operators generate a group, called the Weyl-Heisenberg group 

(1) e= {(6,i^,(^) G M X M X [0,1[} , 
with group multiplication 

(2) (6, z/, if) (b', u', ip') = {b + b',iy + u', cp + cp' - u'b) . 

The specific quotient space P = HI/[0, 1] of the Weyl-Heisenberg group is called 
phase space, or time-frequency plane, which plays a central role in the subsequent 
analysis. Details on the Weyl-Heisenberg group and the time-frequency plane may 
be found in [211 El]- In the current article, we shall limit ourselves to the basic 
irreducible unitary representation of EI on L^(M), denoted by vr", and defined by 

(3) 7r°(6,z/,y.) =e2*"^M,T, . 

By 7r(fe, z/) = Tr°{b, v, 0) we denote the restriction to the phase space. We refer 
to [TT] or [211 Chapter 9] for a more detailed analysis of this quotient operation. 

The left-regular (and right-regular) representation generally plays a central role 
in group representation theory. By unimodularity of the Weyl-Heisenberg group, 
its left and right regular representations coincide. We thus focus on the left-regular 
one, acting on L^(EI) and defined by 

(4) {L{U, v', ^')F\ {b, u, cp) = Fib -b',u- u', ip-ip' + b\p - u')) . 

Denote by /i the Haar measure. Given F,G E L^(H[, (i/i), the associated (left) 
convolution product is the bounded function F * G, given by 



{F*G)ib,u,^)= / Fih)[Lib,u,ip)G]ih)df,ih). 
Jm 

After quotienting out the phase term, this yields the twisted convolution on 

/•oo roo 

(5) (F^G) (b, u)= / F{b', u')G{b -b',u- zy')e-2-6'(---') db'du' . 



The twisted convolution, which admits a nice interpretation in terms of group 
Plancherel theory [11] is non- commutative (which refiects the non-Abelianess of H) 
but associative. It satisfies the usual Young inequalities, but is in some sense nicer 
than the usual convolution, since {E?) C (see [21] for details). 

As explained in [2S1 [2Z] (see also [22] for a review), the representation tt° is 
unitarily equivalent to a subrepresentation of the left regular representation. The 
representation coefficient is given by a variant of the short time Fourier transform 
(STFT), which we define next. 

Definition 1. Lei ^ G L2(R), ^ ^ 0. The STFT of any f e L2(M) is the function 
on the phase space P defined by 

poo 

(6) yj{b, u) = (/, n{b, u)g) = / f{t)g{t - b)e-^'-^' dt . 
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This STFT is obtained by quotienting out [0, 1] in the group transform 

(7) y;f{b,u,^) = {f,7r"{b,iy,^)9) . 

The integral transform intertwines L and n°, i.e. L{h)Yg° = Yg°7r°{h) for all 
/i G H. The latter relation still holds true (up to a phase factor) when 7r° and 
are replaced with vr and Yg respectively. 

It follows from the general theory of square-integrable representations that for 
any g G L^(]R), g ^ 0, the transform Yg° is (a multiple of) an isometry L^(]R) 
L^(P), and thus left invertible by the adjoint transform (up to a constant factor). 
More precisely, given h G L2(M) such that (^f, h) ^ 0, one has for all / G L (M) 

(8) / = 7r^ / 1/J{h,v)T,{h,v)hdhdv . 

\h,g) ip 

We refer to [Bl [21] for more details on the STFT and signal processing applications. 

The STFT, being a continuous transform, is not well adapted for numerical 
calculations, and, for practical issues, is replaced by the Gabor transform, which is 
a sampled version of it. To fix notation, we outline some steps of the Gabor frame 
theory and refer to [HIIM] for a detailed account. 

Definition 2 (Gabor transform). Given g G L^(]R) and two constants bo, uq G M"*", 
the corresponding Gabor transform associates with any f G L^(M) the sequence of 
Gabor coefficients 

(9) ygf{mbo,niyo) = {f , M^^J^mb^g) = (/,5'mn) , 

where the functions gmn = MnuoTmboQ ore the Gabor atoms associated to g and the 
lattice constants b^^VQ. 

Whenever the Gabor atoms associated to g and the given lattice A = b^L x z/gZ 
form a frame j3 the Gabor transform is left invertible, and there exists h G L^(]R) 
such that any / G L^(R) may be expanded as 

(10) = X] '^/("^^o, ni^o)hmn ■ 

m.n 

2.2. The Gelfand triple (5*0, L^, ^q). We next set up a framework for the exact 
description of operators we are interested in. In fact, by their property of being 
compact operators, the Hilbert space of Hilbert-Schmidt operators turns out to be 
far too restrictive to contain most operators of practical interest, starting from the 
identity. Although the classical triple (^5^, L^, S^') might seem to be the appropriate 
choice of generalization, we prefer to resort to the Gelfand triple (5*0, L^, Sq), which 
has proved to be more adapted to a time-frequency environment. Additionally, the 
Banach space property of 5*0 guarantees a technically less elaborate account. 

Definition 3 (5*0). Let ^(M) denote the Schwartz class. Fix a non-zero "window" 
function (p G S^{M.). The space 5*0(1^) is given by 

SoiR) = {fe V{R) : ll/IUo := IKfWmu^) < oo}. 

""^The operator 

is the frame operator corresponding to g and the lattice defined by (5o, vq). If Sg is invertible on 
ij^(M), the family of time-frequency shifted atoms MmboTni^gg, m,n G Z, is a Gabor frame for 
L2(M). 
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The following proposition summarizes some properties of 5*0 (M) and its dual, the 
distribution space Sq(M.). 

Proposition 1. S'o(M) is a Banach space and densely embedded in L^(M). The 

definition of So{E^ is independent of the window ip G =5^(M), and different choices 
of if E ^(M) yield equivalent norms on S'o(M). 

By duality, L^(M) is densely and weak* -continuously embedded in Sq(R) and can 
also be characterized by the norm = ||V(p/||l°°- 

In other words, the three spaces (S'o(M), L^(]R), S'q(M)) Represent a special case 
of a G elf and triple [23] or Rigged Hilbert space. For a proof, equivalent character- 
izations, and more results on 5*0 we refer to [131 [121 ESj • 

Via an isomorphism between integral kernels in the Banach spaces 5*0, 5*0 and the 
operator spaces of bounded operators S'q i— > Sq and 5*0 ^ S'q, we obtain, together 
with the Hilbert space of Hilbert-Schmidt operators, a Gelfand triple of operator 
spaces, as follows. We denote by J3§ the family of operators that are bounded 
Sq and by the family of operators that are bounded Sq ^ Sq. We have the 
following correspondence between these operator classes and their integral kernels 
K,: 

H e {SS, M', SS') ^ — >Ku^ (5o(K), L2(M), . 

We will make use of the principle of unitary Gelfand triple isomorphisms, described 
for the Gelfand triples just introduced in [16]. The basic idea is the extension of a 
unitary isomorphism between L^-spaces to isomorphisms between the spaces form- 
ing the Gelfand triple. In fact, it may be shown, that it suffices to verify unitarity 
of a given isomorphic operator on the (dense) subspace 5*0 in order to obtain a 
unitary Gelfand triple isomorphism, see [161 Corollary 7.3.4]. The most prominent 
examples for a unitary Gelfand triple isomorphism are the Fourier transform and 
the partial Fourier transform. For all further details on the Gelfand triples just 
introduced, we again refer to [TB], only mentioning here, that one important reason 
for investigating operator representations on the level of Gelfand triples instead of 
just a Hilbert space framework is the fact, that 5*0 contains distributions such as 
the Dirac functionals. Shah distributions or just pure frequencies and contains 
operators of great importance in signal processing, e.g. convolution, the identity 
or just time-frequency shifts. 

Subsequently, we will usually assume that the analysis and synthesis windows 
g, h are in 5*0. This is a rather mild condition, which has almost become the canon- 
ical choice in Gabor analysis, for many good reasons. Among others, this choice 
guarantees a beautiful correspondence between the ^^-spaces and corresponding 
modulation space [23]. In the £^-case this means, that the sequence of Gabor 
atoms generated from time frequency translates of an 5*0 window on an arbitrary 
lattice A is automatically a Bessel sequence (in such a case, the window is termed 
"Bessel atom"), which is not true for general L^-windows. 

The Banach spaces 5*0 and Sq may also be interpreted as Wiener amalgam 
spaces [121 Section 3.2.2]. These time-frequency homogeneous spaces are defined 
as follows. Let JFL^ denote the Fourier image of integrable functions and let a 
compact function (p G JFL^(]R) with '^„^^4'{x — n) = 1 be given. Then, for 
X(M) = JFL-^(]R) or X(]R) = C(M), i.e. the space of continuous functions on M, or 
any of the Lesbesgue spaces, we define, for p G [1, oo), with the usual modification 
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for p = oo: 

(11) W(X,n = {fe X,„, : l|/l|w(x/p) = ( E ||/T.0||^)'/" < 00} 

n&Z 

Now, ^0 = W{TL\e) and S'^ = W(J^L°°, £°°), see O Section 3.2.2]. 

2.3. The spreading function representation and its connections to the 

STFT. The so-called spreading function representation, closely related to the inte- 
grated Schrodinger representation [211 Section 9.2], expresses operators in (=^, J^, 
as a sum of time-frequency shifts. More precisely, one has (see Chapter 9]): 

Theorem 1. Let H G {^,^,^'); then there exists a spreading function rj^ in 
(5o(M2),L2(R2),S'^(M2)) such that 

/OO noo 
I 7]H{b,u)n{b,iy)dbdiy . 
00 J —00 

For H G 7i, the correspondence H ^ tjh is isometric, i.e. \\H\\,jf' = ||?7//||l2(p)- 

Remark 1. For H & the decomposition given in 0121) is absolutely convergent, 
whereas, for H G it holds in the weak sense of bilinear forms on Sq. 
When rjH G L2(P), if is a Hilbert-Schmidt operator, and the above integral is 
defined Bochner integral. 

The spreading function is intimately related to the integral kernel k = kh of H 
via 

/oo /"OO 
KHit,t-b)e-^''"'^dt and Knit, s) = / 7]H{t - s, iy)e^''"'Uiy 
00 J ~oo 

As a consequence, for kh G {Sq, L^, S'q), we also have r]H G {Sq, L^, S'q). In partic- 
ular, this leads to the following expression for a weak evaluation of Gelfand triple 
operators: 

(13) {K, L)(^a^^^aj,) = {k{K), k(L))^So,l^s',) = {viK),r]{L))^So,L^si,) 

For L = g® f*, i.e. the tensor product with kernel k(s, t) = g{s)f(t) and spreading 
function 77(6, u) = i^gfib, u), we thus have: 

(14) (/^,(7®r)(,^,^,^o = {K{K),Kig®f*))^So,i.^s^) = {v{K),'rj)^s,,j^2^s',) 

Let us also mention that the spreading function is related to the operator's Kohn- 
Nirenberg symbol via a symplectic Fourier transform, which we define for later 
reference. 

Definition 4. The symplectic Fourier transform is formally defined by 

jr^F(t,0= / F{b,v)e-^^'^^^^"'^dbdv 
Jp 

The symplectic Fourier transform is a self-inverse unitary automorphism of the 
Gelfand triple [Sq, L^, S'q). We will make use of the following relation. 

Lemma 1. Assume that fi, f2, gi, g2 ^ L2(]R). Then 
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Proof: The analoguous statement for the conventional (Cartesian) Fourier trans- 
form reads T{'fg^fi'yg^f2){x,uj) = {yf^f2yg^gi){—uj,x) and has been shown in [25| 
Lemma 2.3.2]. The fact, that 

[TsF]{x,uj)=F{-uj,x) 

completes the argument. 4^ 
Recall that the spreading function of the product of operators corresponds to the 
twisted convolution of the operators' spreading function. Assume Ki in (=^^, J^, SS'^ 
and K2 in , ^) then 



(15) rj{K2-Ki)=v{K2)\\v{K,). 

The spreading function representation of operators provides an interesting time- 
frequency implementation for operators, stated in the following proposition. It 
turns out to be closely connected to the tools described in the previous section, in 
particular twisted convolution and STFT. 

Proposition 2. Let H be in J^, and let rj = rjH be its spreading function 
m (S'o(M2),L2(R2),5^(M2), g g Sq{M), then the STFT of H f is given by a 

twisted convolution of rjH and i^gf: 

YgHfiz) = M-rgfKz). 

Proof: By f|T2l) . we may write 

YgHfiz') = {Hf,n{z')g) 

(16) = j{r^H{z)7riz)f,niz')g)dz 
= J r^H{z){f,7r{zy7r{z')g)dz 

= I r^H{z)e-'-''^'''-^\f,n{z'-z)g)dz 

= I MzWiz' - z)e-'^''^^'-^^dz = {tihWMz') 

Note that 5*0 is time-frequency shift-invariant, so 'i^{z)g is in Sq for all z. Hence, 
the expression in (fT6|) is well-defined. 

If / e L2(M) and H e ,3^ , then Ygf, YgHf and rju E V(M.'^), which is in accordance 
with the fact that L^t^L^ C L^. 

If / G 5*0 (M), then H may be in such that r]H G ^^(M^), hence Hf G S'^(M). 
Hence, we have Ygf G 5o(M^) and -TgHf G 5o(M^). This leads to the inclusion 
Sq\]W{C,£^) C L°°, which may easily be verified directly. 

On the other hand, if / G S'^iR) and H in ^, such that rin G S'o(M^), then Hf 
is in So(R). Hence, Ygf G S'q(R'^) and YgHf G So{M.'^). Here, this leads to the 
conclusion that we have, for / G 5*0 (M): 

(17) SoW ^ So. 

Although it is known that Ygf is not only L°°(M^), but also in the Amalgam space 
W(J^L\£°°) for / G S'o(R) and g G S'o(M),[I2], it is not clear, whether ([T7D also 
holds for functions F G W(jFL^,^°°), which are not in the range of 5'g(M) under 
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'Yg. This and other interesting open questions concerning the twisted convolution 
of function spaces are currently under investigatioiJl. 

Remark 2. As a consequence of the last proposition, H may be realized as a twisted 
convolution in the time-frequency domain: 

/oo /"OO 
/ {h,v)M,mdhdv for all / G {S^.\^\S'^). 

-oo J — oo 

Notice that Proposition [2] implies that the range of is invariant under left 
twisted convolution. Notice also that this is no longer true if the left twisted 
convolution is replaced with the right twisted convolution. Indeed, in such a case, 
one has 

Hence, one has the following simple rule: left twisted convolution on the STFT 
amounts to acting on the analyzed function /, while right twisted convolution on 
the STFT amounts to acting on the analysis window g. It is worth noticing that 
in such a case, applying to ygfc^riH yields the analyzed function /, up to some 
(possibly vanishing) constant factor. 

Example 1. As an illustrating example, let g,h E Sq be such that {g,h) = 1 
and consider the oblique projection P : f ^ {f,g)h. The spreading function 
of this operator is given by Ygh, and we have Y^Pfi^z) = {f,g){h,'7i{z)ip). By 
virtue of the inversion formula for the STFT, which may be written as {f,g)h = 
J {h, 7r{z)g)TT{z)fdz, we obtain: 

y^Pfiz) = J {h,7riz)g){7riz)f,7riz')^)dz = tgh^t^f. 

By completely analogous reasoning, we obtain the converse formula, if the operator 
is applied to the analysing window: 

rp^f = {f,7,{z)p^) = r^f^ygh. 

Remark 3. Notice also that twisted convolution in the phase space is associated 
with the true translation structure. Indeed, time-frequency shifts take the form of 
twisted convolutions with a Dirac distribution on P: 

This corresponds to the usage of engineers, who "adjust the phases" after shifting 
STFT coefficients [I0l[20]. 

2.4. Time-Frequency multipliers. Section 12.31 has shown the close connection 
between the spreading function representation of Hilbert-Schmidt operators and 
the short time Fourier transform. However, the twisted convolution representation 
is generally of poor practical interest in the continuous case, because it does not 
discretize well. Even in the finite case, it relies on the full STFT on C^, which rep- 
resents vectors with A^^ STFT coefficients, which may be far too large in practice, 
and sub-sampling is not possible in a straightforward way. 

Time-frequency (in particular Gabor) multipliers represent a valuable alterna- 
tive for time-frequency operator representation (see [T71 [28] and references therein 
for reviews). We analyze below the connections between these representations and 



H. Feichtinger and F. Lucf. Twisted convolution properties for Wiener amalgam spaces. In 
preparation, 2008. 



OPERATORS IN TIME-FREQUENCY DOMAIN 



9 



the spreading function, and point out some limitations, before turning to general- 
izations. 

2.4.1. Definitions and main properties. Let g,h & 5'o(M) be such that {g,h) = 1, 
let m e L°°(M2)^ and define the STFT multiplier M^.g^t by 

(19) M^,g,J = [ m(6, iy)yj{b, u) Ti{h, u)hdbdu, 

This defines a bounded operator on (5*0 (M), L^(ffi), 50(1^)). 

Similarly, given lattice constants bo, uq G M"*", set Timn = T^ijinho, tiuq) = MnuoTmbo- 
Then, for m G £°°(Z^), the corresponding Gabor multiplier is defined as 

oo oo 

(20) Mg.g,hf=Y. J2 '^^'^^''^^3fi^bo,nuo)7T^^h . 

m=— oo n=— oo 

Note that Gabor multipliers may be interpreted as STFT multipliers with multiplier 
m in S'q. In fact, in this case, m is simply a sum of weighted Dirac impulses on 
the sampling lattice. 

The definition of time- frequency multipliers can of course be given for g,h E 
L^(M), many nice properties only apply with additional assumptions on the win- 
dows. Abstract properties of such multipliers have been studied extensively, and we 
refer to [17] for a review. One may show for example that, whenever the windows 
g and h are at least in Sq, if m belongs to L^(P) (or ^^(Z^)) then the corresponding 
multiplier is a Hilbert-Schmidt operator and maps 5*0 (M) to L^(]R). 

The spreading function of time-frequency multipliers may be computed explicitly. 

Lemma 2. The spreading function of the STFT multiplier Mm;g,/i is given by 

(21) VM^.^^,^ib,u) = .£{b,i.)yghib,u) , 

where ^ is the symplectic Fourier transform of the transfer function m 

^(t,0= ! m{b,u)e^'^^'''~^^Ubdu . 

Specifying to the Gabor multiplier Wl^.^ ^, we see the same expression for the spread- 
ing function, however, in this case, ^ = ^^'^^ is the {uq~^ ^b^^^) -periodic symplec- 
tic Fourier transform of the discrete transfer function m 

oo oo 

(22) ^W(t,^)= Y m{m,n)e^'^^'"'''-'^^°^K 

m=—oo n=—OD 

Proof : For f E Sq and (p E Sq, we may write 

where the right-hand side inner product has to be interpreted as an integral or 
infinite sum, respectively. By Lemma [H applying the symplectic Fourier transform, 
we obtain 

By calling on (fill) , this proofs ([21]) • By virtue of that fact that for g,h E Sq, 
Ygh is certainly in L^(P) and even in the Wiener Amalgam Space W{C, L^), hence 
in particular continuous, the expressions for the spreading function given in the 
lemma are always well-defined. The symplectic Fourier transform is a Gelfand 
triple isomorphism of (S'q, L^, S'q), i.e., m G (S'q, L^, S'q) <^==^ ^ G (S'q, L^, S'q). 
Hence, ^ ■ "fgh G (S'q, L^, Sq), which is in accordance with the fact, that for m in 
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{i^yi"^, i.e., {So, L^, S'q)(Z^), the kernel of the resuhing operator (and hence its 
spreading function), is in {So, L^, Sq), see [17] for details. ^ 

Remark 4. All expressions derived so far are easily generalized to Gabor frames for 
Mf^ associated to arbitrary lattices A C M^*^. In such situations, the spreading func- 
tion takes a similar form, and involves some discrete symplectic Fourier transform 
of the transfer function m, which is in that case a A°-periodic function, A° being 
the adjoint lattice of A, see Definition [7] in Section [31 

Notice that as a consequence of Theorem [1], one has the following "intertwining 
property" 

YgMf = {^^,^,j,ygh)^yj . 

Remark 5. It is clear from the above calculations that a general Hilbert-Schmidt 
operator may not be well represented by a TF-multiplier. For example, let us 
assume that the analysis and synthesis windows have been chosen, and let r] be the 
spreading function of the operator under consideration. 

• In the STFT case, if the analysis and synthesis windows are fixed, the 
decay of the spreading function has to be fast enough (at least as fast as 
the decay of i^gh) to ensure the boundedness of the quotient ^ = rj/'i^gh. 
Such considerations have led to the introduction of the notion of underspread 
operators [21] whose spreading function is compactly supported in a domain 
of small enough area. A more precise definition of underspread operators 
will be given below. 

• In the Gabor case, the periodicity of ^'^'^^ imposes extra constraints on 
the spreading function r]. In particular, the shape of the support of the 
spreading function must influence the choice of optimal parameters for the 
approximation by a Gabor multiplier, i.e. the shape of the window as well 
as the lattice parameters. The following numerical example indicates the 
direction for the choice of parameters in the approximation of operators by 
Gabor multipliers. 

Example 2. Consider two operators OPl and 0P2 with spreading functions as 
shown in Figure [U The values of the spreading functions are random and real, 
uniformly distributed in [—0.5,0.5]. 

Operator 1 has a spreading function with smaller support on the time-axis, which 
means that the corresponding operator exhibits time-shifts across smaller intervals 
than Operator 2, whose spreading function is, on the other hand, less extended in 
frequency. The effect in the opposite direction is, obviously, reverse. These char- 
acteristics are illustrated by applying the operators to a sinusoid with frequency 1 
and a Dirac impulse at —1, respectively. 

Next, we realize approximatioiu by Gabor multipliers with two fixed pairs of lattice 
constants: 6i = 2, i^i = 8 and 62 = 8, z/2 = 2. Furthermore, the windows are Gauss- 
ian windows varying from wide (j = 0) to narrow (j = 100). Thus, j corresponds to 
the concentration of the window, in other words, j is the reciprocal of the standard 
deviation. Now the approximation quality is investigated. The results are shown 
in the lower plots of Figure [H where the left subplot shows the approximation 
quality for operator OPl for hi, Vi (solid) and 62, 1^2 (dashed), while the right hand 
subplot gives the corresponding results for operator 2. The error is measured by 
err = \\0P — APP\\,^^/\\OP + APP\\j^. Here, APP denotes the approximation 



'Best approximation is realized in Hilbert Schmidt sense, see the next section for details. 
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Spreading function 0P1 Spreading function 0P2 




Figure 1. Approximation by Gabor Multipliers with parameters 
hi.ui (solid) and b2, 1^2 (dashed) 



operator and the norm is the operator norm. The results show that, as expected, 
the "adapted" choice of time-frequency parameters leads to more favorable ap- 
proximation quality. Here, the adapted choice of b and u mimics the shape of the 
support of the spreading function according to formula fl2T]) and the periodicity of 
In brief, if the operator realizes frequency-shifts in a wider range, we will 
need more sampling-points in frequency and vice-versa. It is also visible, that the 
shape of the window has considerable influence on the approximation quality. 

The previous example shows, that the parameters in the approximation by Gabor 
multipliers must be carefully chosen. Let us point out that the approximation 
quality achieved in the experiment described in Example [2] is not satisfactory, 
especially when the time- and frequency shift parameters are not well adapted. 
Operators with a spreading function that is not well-concentrated around 0, i.e. 
"overspread operators", don't seem to be well-represented by a Gabor multiplier 
even with high redundancy (the redundancy used in the example is 8). Moreover, a 
realistic operator will have a spreading function with a much more complex shape. 
The next section will give some more details on approximation by Gabor multipliers 
before generalizations, which allow for approximation of more complex operators, 
are suggested. 
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2.4.2. Approximation by Gabor multipliers. The possibility of approximating oper- 
ators by Gabor multipliers in Hilbert-Sclimidt sense depends on the properties of 
the rank one operators associated with time-frequency shifted copies of the analysis 
and synthesis windows. 

Let g,h ^ So{M.) be such that {g,h) = 1. Let A = z/i) e P, and consider the 
rank one operator (oblique projection) Px defined by 

(23) Pxf = {gl ® hx)f = (/, gx)hx , f e {So{R), L\R), S',{m)) . 
Direct calculations show that the kernel of Pa is given by 

(24) ^ip^{t,s)=gxis)hx{t) , 
and its spreading function reads 

(25) r]p,{b,u)=e^'^^''^'~'^''^%hib,iy) . 

The following result characterizes the situations for which time-frequency rank 
one operators form a Riesz sequence, in which case the best approximation by a 
Hilbert- Schmidt operator is well-defined. This result first appeared in [H]. Here, 
we give a slightly different version, which is obtained from the original statement by 
applying Poisson summation formula. This result was also given in [B] for general 
full-rank lattices in M.'^. 

Proposition 3. Let g,h E L^(M) , with {g, h) ^ 0, let bo,i'o E M"*", and set 



(26) U{t,0= E ^9h(t+-,^+-pj 

The family {Pmbo,nuo: rn.n eIj} is a Riesz sequence in H if and only if there exist 
real constants < A < B < (yo such that 

(27) < A<U{t,0 < B < oo a.e. on [0,uo~^[x[0,bo~^[. 

We call this condition the U condition. 

It turns out, that the approximation of a given operator via a standard minimiza- 
tion process yields an expression, which is only well-defined if the U condition ([2] 
holds. 



Theorem 2. Assume that i^gh and bo, uq E are such that the U condition [2l\ ) is 
fulfilled. Then the best Gabor multiplier approximation (in Hilbert- Schmidt sense) 
of H E is defined by the time-frequency transfer function m whose discrete 
symplectic Fourier transform reads 



, Y.Z=^^'y,h{b^klvo,v^ilbo)r^H{h + klvo,v^ilb,) 
28 M[b,v] = — ^ 

Proof: Let us denote as before by □ the rectangle □ = [0, z/o~^[x [0, 6o~^[! and 
set = "i^gh for simplicity of notation. First, notice that if rjH E Iv^(P), then the 
function (6, u) E D ^ J2ke Ivnib + k/uo, v + ^/&o)P is in L^(n), and is therefore 
well defined almost everywhere in □. Thus, by Cauchy-Schwarz inequality, the 
numerator in fl28|l is well-defined a.e. 

The Hilbert-Schmidt optimization is equivalent to the problem 

min \r\ii - J^-ff . 
.^eL2(n) 
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oo />oo 



oo J — oo 

oo 



J2 \VH{b + k/uo,iy + i/bo)-^{b,u)y{b + k/uo,u + i/bo)f dbdu 

,£=—oo ^ 

// \y2\VH{b + k/i,o,i^ + i/bo)\' 

- 23? {^{b, v) J2vH{b + k/uo, V + ijb^fF^b + kjv^, V + £/6o) 



dbdv 



From this expression, the Euler-Lagrange equations may be obtained, which read 
^(6, v)Y,\y{Mlv^,v^ilb^)\^ = Y,VH{b+k/uo,u+i/bo)y{b+k/uo,u+i/bo) , 



kl 



and the resuh follows. 4|k 
We next derive an error estimate for the approximation. Let us set, for [b, u) G □, 



S{b,u) 



J2k e VHib + k/uo, V + l/bQ)'r{b + /c/z/q, ^ + 



Corollary 1. With the above notation, we obtain the estimate 



Imfy^^ < llr/nf ||1-^| 



for the best approximation of H by a Gabor multiplier Mm according to 

Proof : Set Tuip, u) = J2ke IVH^b + k/u^, v + £/6o)P- Replacing the expression 
for ^ obtained in ( l28l) into the error term, we have 



\H 




\^H[b,v) - \^{b,u)\^U{b,u)] dbdv 
j j TH{b,y) [l~£{b,u)] dbdu 



< Wrjuf ||1 - ^^lloo , 

where we have used the fact that ||r//||L2(n) = ||?7h||l2(p)- Clearly, Cauchy-Schwarz 
inequality gives \S{b, z/)| < 1 on □, with equality if and only if there exists a function 
(f) such that ?7 = (p'V, i.e. if and only if if is a multiplier with the prescribed window 
functions. Hence we obtain 



(29) 



\H 



1 — ess inf S(b, u) 

ib,u)eO 



Remark 6. £{b^v) essentially represents the cosine of the angle between vectors 
{r]H{b + k/vQ, V + £/6o), A;, £ G Z} and {f{b + kjv^, v + £/6o), k,i ^ Z}. In other 
words, the closer to colinear these vectors, the better the approximation. 
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An example for operators which are poorly represented by this class of multipliers 
are those with a spreading function that is not "well-concentrated" . These are, in 
technical terms, overspread operators. The underspread/overspread terminology 
seems to originate from the context of time-varying multipath wave propagation 
channels [30]. However, different definitions exist in the literature. Here, we give 
the definition used in [32]. Note that underspread operators have recently found 
renewed interest [331 El ESI El EH] • 

Definition 5. Consider an operator K with compactly supported spreading func- 
tion: 

suppijiK) C (5(to,^o), where Q{to,^o) ■= Ho,^o] x [-^o,^o]- 
Then, K is called underspread, if t^^Q < 1/4. 

Most generally, operators which are not underspread, will be called overspread. 
It is generally known, that an operator must be underspread in order to be well- 
approximated by Gabor multipliers. Formula (l28l) enables us to make this state- 
ment more precise. 

Corollary 2. Consider an underspread operator H . Then, for (60,^^0) such that 
1/60 > 2,^0 f*^'^ 1/^0 > 2to> it is possible to find a Cabor frame {g„in,m,n G Z}, 
with lattice constants {bQ^Vo), and dual window h. Then, the symplectic Fourier 
transform ^ of the time-frequency transfer function of the best Cabor multiplier 
takes the form 



U 

and the approximation error can be bounded by 



Proof: 



VlH - 



\t]h - Jlfgh\ 



h\?< 



vim 



ess sup 

(6,!^)en 



1 - 



U{b, v) 




□ 



< WtihW ess sup 



1 - 



\7^{h,v)%h{h,v)\' 

U{b, v) 
U{b, v) 



dbdu 



The estimate in the last corollary shows, that approximation quality is a joint prop- 
erty of window and lattice, which is in accordance with the results of Example [H 

Remark 7. Note that, although technically only defined for Hilbert-Schmidt opera- 
tors, the approximation by Gabor multipliers can formally be extended to operators 
from see [T71 Section 5.8]. Also, the expression given in fl25]l is well-defined in 
S'q whenever rjn is at least in S'q. However, for non-Hilbert-Schmidt operators, it 
is not clear, in which sense the resulting Gabor multiplier represents the original 
operator. The following example shows that at least in some cases, the result is 
however the intuitively expected one. 

Example 3. Consider the operator vr(A), i.e. a time-frequency shift. Although this 
operator is clearly not a Hilbert-Schmidt operator, we may consider its approxima- 
tion by a Gabor-multiplier according to (128|) . First note that the spreading function 
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of the time-frequency shift 7r(A) = 7i{bi, ui) is given by rj^, = 5{h — hi) ■ 5{u — ui). 
Then, we have 



fghib + k/uo, V + llho)b{\) - 61 + /c/z/q, v-vx^ l/h) 



Ek',i'Wb + k'/iyo,^ + l'/bo)\' 
6{b-bi + k/uo, V - vi + I /ho) 

fc'-fc I l'-l\\2 



^,tf''T E - ^1 + ^/^O' - + v^o) 

Hence, from the inverse (discrete) symplectic Fourier transform we obtain: 



m(m,n) = ^^^^^ V - 61 + A;/z.o, - z.^ + //M 

Uipi.ui) Jo Jo ^ 

^^-2ni{mbou-nuob) ^^^^ 
_ %h{bi, Z/i) 2ni{mbQUi-nuobi) 

U{h,vx) 

As expected, the absolute value of the mask is constant and the phase depends 
on the displacement of A from the origin. This confirms the key role played by 
the phase of the mask of a Gabor multiplier. Specializing to A = 0, we obtain a 
constant mask and thus, if /i is a dual window of g with respect to A = ho'L x z/qZ, 
up to a constant factor, the identity. 

3. Generalizations: multiple Gabor multipliers and TST spreading 

functions 

In the last section it has become clear that most operators are not well repre- 
sented STFT or Gabor multiplier. 

Guided by the desire to extend the good approximation quality that Gabor mul- 
tipliers warrant for underspread operators to the class of their overspread counter- 
parts, we introduce generalized TF-multipliers. The basic idea is to allow for an 
extended scheme in the synthesis part of the operator: instead of using just one 
window /i, we suggest the use of a set of windows {h^^^} in order to obtain the class 
of Multiple Gabor Multipliers (MGM for short). 

Definition 6 (Multiple Gabor Multipliers). Let g G >S'o(M) and a family of recon- 
struction windows h^^'> G S'o(M), j G JT", as well as corresponding masks uij G i"^ 
be given. Operators of the form 



(30) 



will be called Multiple Gabor Multipliers (MGM for short). 

Note, that we need to impose additional assumptions in order to obtain a well- 
defined operator. For example, we may assume J2j ^'^P^^l^jWl = C < 00 and 
maxj ll/i-'lls'o = C < 00, which guarantees a bounded operator on {So, L^, S'q). This 
follows easily from the boundedness of a Gabor multiplier under the condition that 
m is Conditions for function space membership of MGMs are easily derived 
in analogy to the Gabor multiplier case. For example, if ^ . |mj(A)p < 00, we 
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obtain a Hilbert-Schmidt operator, similarly, trace-class membership follows from 
an analogous £^-condition. 

As a starting point, we give the (trivial) generalization of the spreading function 
of a MGM as a sum of the spreading functions corresponding to the single Gabor 
multipliers involved. 

Lemma 3. The spreading function of a MGM is (formally) given by 
(31) r,^ {b,u) =Y,Ji^^\h,v)'y,h^^[h,v) , 

where the (vq^^ ,bQ~^) -periodic functions ^^^^ are the symplectic Fourier trans- 
forms of the transfer functions uij . 

Note that the issue of convergence for the series defining r]^ (6, u) will not 

be discussed, as in practice \ J\ will usually be finite. Let us just mention that 
by assuming m^ G £^(Z'^), i.e., the Hilbert-Schmidt case with an additional 
condition for the masks in the general model, we have ||?7^ , ||2 = C||m||2. 



It is immediately obvious that this new model gives much more freedom in gener- 
ating overspread operators. However, in order to obtain structural results, we will 
have to impose further specifications. 

Before doing so, we will state a generalization of Proposition [3] to the more general 
situation of the family of projection operators defined by 

(32) Pif = {gl ® h{)f = {f,g>)hl where A G A, j G J, \J\ < oo. 

Note that these projection operators are the building blocks for the MGM. The 
following theorem characterizes their Riesz property. 

Proposition 4. Let g, G L^(]R), j G J, \J\ < oo, with {g, h^) ^ 0, let bo, uq G 

M^, and let the matrix r(6, v) be defined by 

(33) r(6, z/),y = Y/r^\b + A;/z/o, z/ + i/bo)%h^''\b + k/uo, v + ^/&o) 

a.e. on □ = [0, z/(^^[x [0, 6q "'^[. Then the family of projection operators {Pl,j G 
Z, \ E A} is a Riesz sequence in H if and only ifV is invertible a.e. 
Alternatively, the Riesz basis property is characterized by invertibility of the matrix 
U defined as 



(34) U"'{t,i) = Y,U"\kb,Mo)e 



2TTi{luQt—kbo^) 
k,l 



a.e. on the fundamental domain of A. 

Proof: Recall that the family {Pl,j G Z, A G A} is a Riesz basis for its closed 
linear span if there exist constants < A, B < oo such that 



(35) Allelic < 



<B\\c\\l 



HS 
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for all finite sequences c defined on (Ax J\ We have 
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A i 



us \ A j j' 

Am jj' 



Hence, by setting U^^' {b, u) = [Ygg ■ 'fhjh^']{b, u), we may write 



A j 



ns 



EE^A4'f/^'^'(/^-A) 

Am jj' 

M jj' 



jj' 



L2(n) 



where U"' {t, ^) = ^ U^^'{kbo, /z/o)e 27r«(«i/ot fcboC) jg ^^j^g discrete symplectic Fourier 

transform of U^^\ and, analogously, C-' is the discrete symplectic Fourier transform 
of the sequence , defined on A, for each j. Hence, these are x 6q ^-periodic 
functions. The last equation can be rewritten as 

EE^Ai^" 



A i 2 

where U is the matrix with entries U^^' . Note that this proves statement (!34|) by 
positivity of the operator U. 

In order to obtain the condition for F given in (1331) . first note that 



J'sifgg ■ )(A) = {Ygh^ ■ fgWm 



by applying Lemma [H Furthermore, F := '^/i-'' ■ '^gh^ is always in for h? G 
\? . We may therefore look at the Fourier coefficients of its A°-periodization, with 
A = (m6o, nvo): 

J^-\PaoF)(\) = [ (y F(b+—,l^ + L))e^-iibnuo-vmb,)^^^^ 



F{b, v)e'^'^^'"'°-'''"'^'^dbdv = 0^:^%^' ■ yghi){\). 

Hence, we may apply the Poisson summation formula, with convergence in L^(n) 
to obtain: 

boyoY^s\%h^' ■W){kboM)e-^^'^'''''~'''''^ 

k,l 
k,l 
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We conclude that 



(36) 



EE 4^' 

i AGA 



boi^o / T{b,iy)jj,C^{b,iy)a'{b,u)dbdiy, 



n 

2 



and the Riesz basis property is equivalent to the invertibility of V. l|k 

In the sequel, the discrete symplectic Fourier transforms of nij will be denoted 
by and the vector with as coordinates will be denoted by ^ . We then 
obtain an expression for the best multiplier in analogy to the Gabor multiplier case 
discussed in Theorem |2l 

Proposition 5. Let g E 5*0(11^) and h^^'^ E Sq(R), j E J he such that for almost all 
b,iy, the matrix T(b, u) defined in (133!) is invertible a.e. on □ = [0, z/(7^[x [0, "'^[. 

Let H E {j3§,Jif,^') be an operator with spreading function rj E {Sq,Ij'^, Sq) . 
Then the functions yielding approximation of the form ^3^) may be obtained 
as 

(37) ^ = ■ i3 , 

where B is the vector whose entries read 

(38) B,,{b, v) = Y. vib + k/uo, v + tlb^fV^W^b + kjv^, v + ijb^). 

For operators in the obtained approximation is optimal in Hilbert- Schmidt sense. 

Proof: The proof follows the lines of the Gabor multiplier case. The optimal 
approximation of the form (!30|) . when it exists, is obtained by minimizing 

lh~5Il^i'^ir = 5^1 / \v{b+k/iyo,J^+^/bo)-^^j{b,u)yj{b+k/uo,u+i/bo)\^ dbdu 

where one has set = Ygh^^l Setting to zero the Gateaux derivative with respect 
to ^j„, we obtain the corresponding variational equation 

^ ^,{b, u) ^ ^(ft + k/uo. y + ^/hoWoAb + k/u^, v + ijb^) = B,,{b, v) , 

3 k/ 

where Bj{b, v) are as defined in fl38|) . Provided that the r(6, v) matrices are invert- 
ible for almost all 6, z/, this implies that the functions for approximation of the 
form fl30l) may indeed be obtained as in (l37|l . 4- 
In a next step, we are going to discern two basic approaches: 

(a) mj{X) = m{fi, A), i.e. the synthesis windows are time-frequency shifted versions 
(on a lattice) of a single synthesis window: hj = 7i{fij)h, fij E Ai. 

(b) mj{\) = mi(A)m2(j), i.e. a separable multiplier function. If we set h^^\t) = 
7r{bj,h'j)h(t) then this approach leads to what will be called TST spreading func- 
tions in Section 13.21 

In both cases we will be especially interested in the situation in which the are 
given as time-frequency shifted versions of a single synthesis window on the adjoint 
lattice A°. 

Definition 7 (Adjoint lattice). For a given lattice A = boZx z/qZ the adjoint lattice 
is given by A° = —Z x ^Z. 

^ f 1^0 bo 
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Note that the adjoint lattice is the dual lattice A"*" with respect to the symplectic 
character. 

3.1. Varying the multiplier: MGM with synthesis windows on the lattice. 

We fix the synthesis windows hj to be time-frequency translates of a fixed window 
function, i.e. 

(39) h^^\t) = 7r{bj, Uj)h{t) = e^'^^'^'hit - h,) . 

We may turn our attention to the projection operators associated to the (Ga- 
bor) families (5',Ai) and (/i,A2). Note that it has been shown by Benedetto and 
Pfander [B] that the family of projection operators {P\i A G A}, as discussed in 
Section [2.41 either forms a Riesz basis or not a frame (for its closed linear span). 
The next corollary shows that, on the other hand, if we use the extended family of 
projection operators {-Pa,/^, (A,/i) G Ai x A2}, where Pa,/^/ = (/? 7r(A)(7)7r(/i)/i, we 
obtain a frame of operators for the space of Hilbert-Schmidt operators, whenever 
((yf, Ai) and (/i, A2) are Gabor frames. This corollary is a special case of Theorem 4.1 
in [5] and Proposition 3.2 in [1]. 

Corollary 3. Let two Gabor frames {g,Ai) and (/?., A2) be given. Then the family 
of projection operators {Px,ii, (A,//) G Ai x A2} form a frame of operators in TL 
and any Hilbert-Schmidt operator H may be expanded as 

H= c(A,/i)PA,M- 

AGAi,/iGA2 

The coefficients are given by c(A,yu) = (iJ, {Px,fi)*) = {H'n'{fi)h,7r{X)g) . 

Note that an analogous statement holds for Riesz sequences. Very recently, it 
has been shown [T], that the converse of Corollary [3] holds true for both frames 
and Riesz bases, i.e. the family of projection operators {Pa,/^, (A,/x) G Ai x A2} 
is a frame (a Riesz basis) for Ti. if and only if the two generating sequences form 
a frame (a Riesz basis) for L^(M). In particular, this leads to the conclusion, 
that the characterization of Riesz sequences given in Proposition H] also yields a 
characterization of frames for L^(M) - it is well known, that {g\,X G A) form a 
Gabor frame if and only if {g^,^ G A°) form a Riesz sequence. We can draw two 
conclusions. 

Corollary 4. Let g G 5'o(]R) and a lattice A = b^L x VqI, be given. 

(a) The Gabor family {g\^ A G A} forms a frame for L^(]R) if and only if the 
matrix 

^mn,m'n'ib,u) = exp ( - 2i7r[m/ Uq^U + Euq - u/bo) - ui' / Uo^U + iuo - u' /bo)]) 

k,e 

X ygg{b — m/i^o + kbo, v — n/bo + Ivq) 
X %9{b - I^Q + ^^0, V - n' /bo + Ivq) 
is, a.e. on □, invertible on i"^. 

(b) In addition, we may state the following "Balian-Low Theorem for the tensor 
products of Gabor frames" : 

A family of projection operators given by {Px,n = gl ® gfi, (A,/i) G A x A°} 
forms a frame for the space of Hilbert-Schmidt operators on L^(M) if and 
only if it forms a Riesz basis. Hence, in this case, g cannot be in S'o(M). 
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Proof: Statement (a) is easily obtained from ( 1331) by observing that 
fgTi^mhQ, nuo)g{b, v) = e-2*™^o,(™o)^^/^(5 _ ^^^^ ^ _ ^^^^ _ 

We then have that {-Pa,/^ = A'a ® fi'^' (-^'Z^) ^ A° x A°} forms a Riesz basis in H 
if and only if Tmn,m'n'ib,v) is invertible. By the converse of Corollary [31 this is 
equivalent to the Riesz property of g^.n G A°, which, in turn, is equivalent to the 
frame property of {g\, A G A} by Ron-Shen duality, see, e.g. [Mj . 
To see (b), note that in this case Pa,;^ is a frame for 7i -v^ {g\^ A) and {g^, A°) form a 
frame {g\, A) and {g^, A°) form a Riesz basis <^ {P\,^i = gl'^Ofi, (A, /i) G A x A°} 
is a Riesz basis for 7i. Furthermore, by the classical Balian-Low theorem, if a Gabor 
system is an L^-frame and at the same time a Riesz sequence (hence an L^-Riesz 
basis), then the generating window g cannot be in S^. ^. 

We may next ask, when the projection operators form a Riesz sequence, if the 
reconstruction windows are TF-shifted versions of a single window h on the adjoint 
lattice of A = BqI, x uqIj. In fact, in this case, the matrix F turns out to enjoy quite 
a simple form. To fix some notation, let 

X -^3/1(6- (A;-m)/z/o,z/- (^-n)/6o), 
and introduce the right twisted convolution operator 

K\{h, v) ■ J^{h, v) J^{h, v)^A{h, v). 

Corollary 5. Let g,h G So as well as bo, uq be given. Furthermore, let h^^^ = 
7r(^, ■^)h. Then the variational equations read 

(40) J^{b,u)^A{b,u) = B{b,u) . 

Hence, if for all &, z/ G M? , the discrete right twisted convolution operator K\ is 
invertible, then the family Px^fj, = gx® g^, (A,yu) G A x A° forms a Riesz sequence 
and the best MGM approximation of an Hilbert- Schmidt operator with spreading 
function rj is given by the family of transfer functions 

^„„(6,z/) = [{K\{b,u)Y^B{b,u) 

where B is given in / f5^) . 

Proof: As in the proof of Corollary HI we may derive the given form of F by direct 
calculation, achieving the final form by noting that for A = b^L x z/gZ, the adjoint 
lattice is given by A° = — Z x t^Z. 4k. 

We close this section with some results of numerical experiments testing the 
approximation quality of MGMs for slowly time- varying systems. (l39l) . 

Example 4. We study the approximation of a (slowly) time-varying operator. The 
operator has been generated by perturbing a time-invariant operator. The spread- 
ing function is shown in the upper display of Figure [21 The signal length is 32, 
time- and frequency-parameters are 60 = 4 and = 4, such that the redundancy 
of the Gabor frame used in the MGM approximation is 2. The approximation 
is then realized in several steps for two different schemes. Scheme 1 adds three 
synthesis window corresponding to a frequency- shift by 4, a time-shift by 4 and a 



^More precisely, g cannot even be in the space of continuous functions in the Wiener space 
W{R), see [Ml Theorem 8.4.1] 
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Figure 2. Approximation by Multiple Gabor Multipliers 

time-frequency-shift by (4,4). The first step 1 calculates the regular Gabor multi- 
plier approximation. Step 2 adds one (only frequency- shift) and so on. The rank 
of the resulting operator families is 64, 128 for both step 2 and step 3 (adding 
either time- or frequency-shift) and 256 (time-shifted, frequency-shifted and time- 
frequency-shifted window added) . The resulting approximation-errors are given by 
the solid line in the lower display of Figure [21 

Scheme 2 considers synthesis windows shifted in time and frequency on the sub- 
lattice generated by a = 8, 6 = 8, the resulting families having rank 64, 256 and 
576. Here, we only plot the results for the case corresponding to three and eight 
additional synthesis windows, respectively. The results are given by the dotted 
line. 

For comparison, an approximation with a regular Gabor multiplier with redun- 
dancy 8, i.e. an approximation family of rank 256, has been performed. The 
approximation error for this situation is the diamond in the middle of the display. 
It is easy to see that, depending on the behavior of the spreading function, different 
schemes perform advantageously for a certain redundancy. Note that for scheme 1, 
the best MGM with the same rank as the regular Gabor multiplier performs better 
than the latter. In the case of the present operator, scheme 2 performs the "wrong" 
time-frequency shifts on the synthesis windows in order to capture important char- 
acteristics of the operator. However, in a different setting, this scheme might be 
favorable (e.g. if an echo with a longer delay is present). 

The example shows, that the choice of an appropriate sampling scheme for the 
synthesis windows is extremely important in order to achieve a good and efficient 
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approximation by MGM. An optimal sampling scheme depends on the analysis 
window's STFT, the lattice used in the analysis and on the behavior of the op- 
erator's spreading function, which reflects the amount of delay and Doppler-shift 
created by the operator. Additionally, structural properties of the family of pro- 
jections operators used in the approximation, based on the results in this section, 
have to be exploited to achieve numerical efficiency. An algorithm for optimization 
of these parameters is currently under development Jl 

3.2. Varying the synthesis window: TST spreading functions. We next 
turn to the special case of separable functions nij(A) = rni{X)m2{j) for the mask 
in the definition fl30|l of MGMs. In this case the resulting operator is of the form 

Mf = 5^mi(A)p(A)(^m2(j)(/ ® h^Mf) = mi(A)p(A)P„/ , 

A j A 

where F^f = J2j ^2(j) (/, g)h^ and p(A) denotes a tensor product of time-frequency 
shifts: 

p{\)H := 7r(A)/J7r*(A). 
Hence, the spreading function of M is given by 

T]M = ^ ■ VP,,,, 

where ^ is the discrete symplectic FT of mi. If the reconstruction windows are 
given by = 7i{fi^)h, fij = {bj, Uj), this becomes 

j 

Motivated by this result, we introduce the following definition. 

Definition 8 (TST spreading functions). Let (p be a given function from the func- 
tion spaces (S'o(M^), L^(]R^), S'o(M^)) and let bi.ui denote positive numbers. Let a 
be in A spreading function r] = tih of H & (e^, Ji^, ^'), that may be written 

as 

(41) r/(6, y) = Y, - z/ - iy,)e~^^<-~'-^)^^- 

will be called Twisted Spline Type function (TST for short). 

Remark 8. By a in the series defining 77 is absolutely convergent in (S'o, L^, S'q). 
For ^^-sequences a, we obtain an L^-function rj for G L^. 

TST functions are nothing but spline type functions (following the terminology 
introduced in [S]), in which usual (Euclidean) translations are replaced with the 
natural (i.e. H-covariant) translations on the phase space P. In fact, by writing 
a(fe, v) = Ylik f.'^ki^^b — kbi, V — ii^i), the TST spreading function may be written 
as a twisted convolution: 77(6, u) = a\](f). This leads to the following property of 
operators associated with TST spreading functions. 

Lemma 4. An operator H G (i^, M'., SS'^ possesses a TST spreading function 
rj G (5*0, L^, 5*0) as in ( f^T] ) if and only if it is of the form 

(42) H^ = Y1 (^k£Tiikbi,£ui)H^ , 

k,l 



P. Balazs, M. Dorfler, F. Jaillet and B. Torrcsani. An optimized sampling scheme for gener- 
alized Gabor multipliers. In preparation, 2008. 
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where is the linear operator with spreading function 0. 

Proof: The proof consists of a straight-forward computation which may be spared 
by noting that we have, by f[T51) : 

Hrj = Ha\i^ = Ha - H^ = ^ aki7i{kbi, iui) . 

k,l 

As before, we are particularly interested in the situation of the synthesis windows 
being given by time- frequency shifted versions of a single window: = 7r{fij)h. In 
a next step we note, that under the condition 7r(A)7r(yUj) = 7r(/ij)7r(A), i.e., fij G A°, 
the MGM with separable multiplier results in a TST spreading function with a 
Gabor multiplier as basic operator H^. 

Lemma 5. Assume that a MGMM with multiplier mj{\) = mi(A)m2(j) is given. 
If the synthesis windows are given by = 7T{fij)h, with fij G A°, then 

j 

i.e., here, the operator H^f, is given by a regular Gabor multiplier with mask mi and 
synthesis window h. 

Remark 9. Comparing the expression in the previous lemma to the expression 
M = mi(A)p(A)Pm for the same operator, we note that in this situation, the 
operator may either be interpreted as a (weighted) sum of Gabor multipliers or 
as a Gabor multiplier with a generalized projection operator Pm in the synthesis 
process. In this situation, we may ask, whether the family of generalized projection 
operators, {p(A)Prn}AGA form a frame or Riesz basis for their linear span. In fact, 
if m is in and g,h G 5*0, this question is easily answered by generalizing the 
result proved in [6l Theorem 3.2]. Here, {p{X)^m}xeA is either a Riesz basis or 
not a frame for its closed linear span. Furthermore, there exists r > such that 
{p(aA)Pni}AeA is a Riesz basis for its closed linear span whenever a > r. 

In generalizing the result of Lemma [5|, it is a natural next step to assume that 
the basic function (p entering in the composition of t] is the spreading function of a 
Gabor multiplier (at least in an approximate sense). According to the discussion 
of Section 12.41 this essentially means that </> is sufficiently well concentrated in the 
time-frequency domain. 

(In the sequel we will write vr^n for 7r{mbo, uvq) whenever the applicable lattice 
constants are sufficiently clear from the context.) 

Hence, we assume that a Gabor multiplier H^, as defined in (l20l) is given. We may 
formally compute 

(43) Hf = '^akeTTke'^m{m,n)ygf{mbo,nuo)7i{mbo,ni^o)h 

k,£ m,n 

= ^ m(m, n)ygf{mbo, nvo) ^ ake7r{kbi, £ui)TT{mbo, nuo)h 

m,n k,£ 

Based on this expression, one may pursue two different choices of the sampling- 
points (kbijiui). First, in extension of the result given in Lemma O we assume 
that the sampling points are associated to the adjoint lattice A° = x of 
A = bo^j X uqIj. The second choice of sampling points on the original lattice leads 
to a construction as introduced in [11] as Gabor twisters and will not be further 
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discussed in the present contribution. 

The following theorem extends the result given in Lemma [5] to the case in which 
the sampling points in the TST expansion are chosen from a lattice containing 
the adjoint lattice. It turns out that the TST spreading function then leads to a 
representation as a sum of Gabor multipliers. 

Theorem 3. Let Bq, uq G generate the time-frequency lattice A, and let A° 
denote the adjoint lattice. Let g,h & «S'o(M) denote respectively Gabor analysis and 
synthesis windows, such that the lA condition l[2l\ ) is fulfilled. Let H denote the 
operator in J^, SS'^ defined by the twisted spline type spreading function r] as 
in Iji41\ ), with hi,Vi G M^. 

(1) Assume that hi and Vi are multiple of the dual lattice constants. Then H 
is a Gabor multiplier, with analysis window g, synthesis window 

(44) 7 = ^aH7r(A;5i,£i^i)/i , 

and transfer function 



(45) m(m, n) = bo^o / ^{b, jy)e~^Mni^ob~m.bou) ^^^^ ^ 

Jn 

with □ the fundamental domain of the adjoint lattice A° , and 



18) ^/ = EE E E m(m,n)r,/(m6o 

t=l i=l m=i[modp] n=j [mod q] 



(46) M[b,v) = ■ ^ 

i:Zi=-oo\%h{b + k/v,,u + E/bo)\' 

(2) Assume that the lattice generated by bi and ui contains the adjoint lattice: 

(47) fci = — , ^1 = ^ ■ 

puQ qbo 

Then H may be written as a finite sum of Gabor multipliers 
p q 

(4J \ 

; [mod p] n=j [mod q] 

with at most p ■ q different synthesis windows ^ij and the transfer function 
given in ([^^ and ([T^- 

Proof: 

Let us formally compute 

Hf = ^rn{m,n)'fgf{mho,nvo)^aki'K{kbi,lvi)-Krnnh 

m,n k,£ 

= ^m{m,n)ygf{mbo,ni^o)7rrnnlmn , 

m,n 

where 

Imn = J]afc,e2-[^-''o-i-^-'^°^iV(A;6i,£i.i)/i 

k,e 

Now observe that if (6i, ui) G A°, one obviously has 

7mn = ^ake7i{kbi,iui)h = 700 , for (m,n) G , 

kJ 
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i.e. the above expression for Hf involves a single synthesis window 7 = 700. 
Therefore, in this case, H takes the form of a standard Gabor multiplier, with 
fixed time-frequency transfer function, and a synthesis window prespribed by the 
coefficients in the TST expansion. This proves the first part of the theorem. 

Let us now assume that the TST expansion of the spreading function is finer than 
the one prescribed by the lattice A°, but nevertheless the lattice Ai = Zbi x Zz/i 
contains A°. In other words, there exist positive integers p, q such that (1471) holds. 
We then have 



EO^^r hnp — lmq -i 
aue PI ^7i{kbi,iui)h 



and it is readily seen that there are at most pq different synthesis windows 7,^, 

(50) 7ij = 7m[modpl,n[modgl ,^ = 1, • • • ,P; J = 1, • • • , 

The operator H may hence be written as a sum of Gabor multipliers, with one 
prescribed time-frequency transfer function, which is sub-sampled on several sub- 
lattices of the lattice A: 

Kj = {pbo -Z + i-bo) X {qvQ ■ Z + j ■ i^o), « = 0, . . . ,p - 1; j = 0, . . . , g - 1, 

and a single synthesis window per sub-lattice as given in ( l50l) . The resulting ex- 
pression for H is hence as given in 



The expression for the transfer function is derived in analogy to the case discussed 
in Section [2^ 

Remark 10. Let us observe that in this approximation, the time- frequency transfer 
function m is completely characterized by the function used in the TST expan- 
sion. The choice of cf) therefore imposes a fixed mask for the multipliers that come 
into play in equation psj) . 



Example 5. We first assume, that for a given primal lattice A = b^L x z/qZ, the 
representation of a spreading function 77 is given by 5 building blocks: 

Vn On 
k=-l ^ l=-l ^ 

In this case, we obtain a single Gabor multiplier with synthesis window 
700 = afco7r(-^, 0)/i + ao£7r(0, . 

Z/n On 

k=-i ^ e=-i ^ 

If we add the windows 0(6 ± z/ ± 2^;^) to the representation of rj, we are now 
dealing with the finer lattice A = tt-Z x t^Z and we obtain the sum of 4 Gabor 
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multipliers with the following synthesis windows: 

1 / r \ 1 



7oo 



7oi 



7io 



711 = 2^afcoe"*'=vr(^— ,Oj/i+2^ao£e-'^*V^O,— )/i 

and corresponding lattices: Aqo = 2Zbo x 2Zz/o, Aqi = 2Z6o x (2Z + l)z/o, Aio = 
(2Z + l)bo X 2Zi^o, and An = (2Z + l)6o x (2Z + l)z/o. 

It is important to note, that in both cases described in Theorem [3] as well as the 
above example, the transfer function m can be calculated as the best approximation 
by a regular Gabor multiplier - a procedure which may be efficiently realized using 
(125]) . Fast algorithms for this exist in the literature, see [13], however, the method 
derived in Section 12.41 appears to be faster. 

4. Conclusions and Perspectives 

Starting from an operator representation in the continuous time-frequency do- 
main via a twisted convolution, we have introduced generalizations of conventional 
time-frequency multipliers in order to overcome the restrictions of this model in the 
approximation of general operators. The model of multiple Gabor multipliers in 
principle allows the representation of any given linear operator. However, in oder 
to achieve computational efficiency as well as insight in the operator's character- 
istics, the parameters used in the model must be carefully chosen. An algorithm 
choosing the optimal sampling points for the family of synthesis windows, based 
on the spreading function, is the topic of ongoing research. On the other hand, the 
model of twisted spline type functions allows the approximation of a given spread- 
ing function and results in an adapted window or family of windows. By refining 
the sampling lattice in the TST approximation, a rather wide class of operators 
should be well-represented. The practicality of this approach has to be shown in 
the context of operators of practical relevance. All the results given in this work 
will also be applied in the context of estimation rather than approximation. 
As a further step of generalization, frame types other than Gabor frames may be 
considered. Surprisingly little is known about wavelet frame multipliers, hence it 
will be interesting to generalize the achieved results to the affine group. 
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